Papers with Self-normalizing discriminative models
Self-Normalization Properties of Language Modeling (C18-1)
Copied to clipboard
| Challenge: | Existing methods to reduce run-times for language models with large word vocabularies are based on noise contrastive estimation (NCE) |
| Approach: | They propose to use noise-constrained noise-based models to approximate the normalized probability of a class without having to compute the partition function. |
| Outcome: | The proposed model outperforms softmax-based models in a variety of NLP tasks and is based on the noise-constrained noise-constant estimation properties. |